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Sequencing and Prior 
Information in Linear 


Programed Instruction 


HERBERT R. MILLER 


During the past eight or ten years, several principles have 
served as guidelines for the authors of linear instructional pro- 
grams. The fact that research on these principles has offered no 
definitive patterns of support does not appear to have had any 
damaging effect on their general acceptance. 

Logical sequencing of content in linear instructional programs 
is the principle under examination in this study. Skinner's 
(1953, p. 169) definition of programing as the ‘‘construction of 
carefully arranged sequences of contingencies leading to the 
terminal performances which are the object of education’ is a 
historical antecedent of this principle and, indeed, the principle 
is still listed commonly among the characteristics of pro- 
gramed instruction. 

Unfortunately, research comparing logically ordered pro- 
gramed sequences with nonlogical, or randomized sequences 
has been unable to lend much support to this principle. Studies 
by Gavurin and Donahue (1961), Roe, Case, and Roe (1962), 
Levin and Baker (1963), Hamilton (1964), Payne, Krathwohl, 
and Gordon (1967), and Miller (1965) were unable to demon- 
strate superiority in achievement for those subjects using logi- 
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cally sequenced programs. Such superiority was demonstrated 
only in a study reported by Roe (1962). Considerable valid 
criticism has been directed against these studies by Holland 
(1965) and others. Areas of criticism included inappropriate 
combinations of program subject matter and learners, poor 
measures, poor programs, length of program, and the nature of 
the random or nonlogical sequences. Indeed, these criticisms 
have led some to dismiss the former six studies in favor of the 
one reported by Roe (1962), even though the Roe study was not 
above criticism. 

Subsequent studies, which have taken into account these crit- 
icisms, have produced similar results, indicating that the ten- 
dency to dismiss earlier results as artifacts of bad design may 
have been premature. A study reported by Niedermeyer, Brown, 
and Sulzen (1968) found no significant differences in achieve- 
ment between three sequence versions (logical, random, and 
reverse order). Wodtke, Brown, Sands, and Fredericks (1968) 
reported a study in which no significant difference in achieve- 
ment was found after use of logical and nonlogical versions of 
a mathematics program which was presumed to have an im- 
portant logical structure. Some evidence, however, was found 
to indicate a relative advantage of logical sequence for lower 
ability learners. 

Consideration of the problem of logical versus nonlogical se- 
quencing in programed instructional materials suggests the 
following questions: | 

1. What effect does sequencing have on the effectiveness of 
a programed unit as measured by criterion and retention tests? 

2. What effect does sequencing have on the efficiency of a 
programed unit as measured by time on the program, and num- 
ber of errors on the program? 

3. Will prior information in the form of an outline specify- 
ing the topics to be covered in the program be helpful to stu- 
dents (especially those using nonlogical sequences)? 

4. Will the experimental variables have an effect on the 
mood or feelings of the students? 

5. How will the students’ level of subject matter achieve- 
ment be related to sequencing and prior information variables? 

The study reported in this paper was designed to suggest an- 
swers to these questions and to take account of as many of the 
criticisms of earlier studies as possible. 
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A 2x4 fully crossed factorial design was used for each of two 
substudies. Eight programs on matrix arithmetic, representing 
all combinations of four methods of sequencing and two levels 
of prior information, were presented to two groups of experi- 
mental subjects. 

Half of the subjects in each substudy received as the first page 
of their programs an outline specifying the definitions and op- 
erations to be covered in the program. The other halt of the sub- 
jects in each substudy did not receive the outline. 

Four different programed sequences were used, including the 
original logical version and three nonlogical versions con- 
structed to present various combinations of disruption of logi- 
cal sequence at two levels of content ordering. The sequencing 
treatments are illustrated in Figure 1. 


Treatment 
Logical Nonlogical Nonlogical Nonlogical 
I II III 
A B B R 
B L A S 
C N C T 
D R D uu 
K A L A 
L U N B 
M M M C 
N T kK D 
“R C R K 
S K U L 
T D T M 
U S S N 
Macro-order Macro-order Macro-order Macro-order 
and and preserved, disrupted, 
micro-order micro-order micro-order micro-order 
preserved. disrupted. disrupted. preserved. 


By designating the order of presentation of the sections of 
the program as macro-order, and the order of development with- 
in the sections as micro-order, the four sequencing treatments 
can be summarized quite briefly. The sequence used in Logical 
preserved both orders. The sequences in Nonlogical I disrupted 
both orders. Those used in Nonlogical II preserved macro-order 
but disrupted micro-order. The sequence used in Nonlogical III 
disrupted macro-order but preserved micro-order. 
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The following is a more complete description of the various 
sequences used. 

1. Logical. This was the standard version of the program, 
which covered matrix arithmetic in three sections: definitions; 
addition of matrices and multiplication by a scalar; and multi- 
plication of two matrices. The logic of the program was that 
imposed by the interactions of the author, subject matter, ob- 
jectives, and learners used in preliminary field trials. Therefore, 
both macro-order and micro-order were preserved. 

2. Nonlogical I. This sequence was prepared by randomiz- 
ing over the entire length of the original sequence, thus dis- 
rupting both macro- and micro-orders. Eight lists of random 
two-digit numbers from one to 96 were prepared as a basis for 
establishing the nonlogical sequences to be used in this treat- 
ment. The three nonlogical sequences having the largest num- 
bers of short runs of content-related frames were then selected 
for use. It was felt that the absence of long runs of content- 
related frames, which can occur by chance in any randomly de- 
termined sequence, would tend to insure maximum disruptive- 
ness of the original logic. The three sequences were used in 
order to minimize the probability that a single nonlogical se- 
quence might actually be ‘logical’ to some of the learners. Each 
of the sequences used in this treatment had zero rank order 
correlations with the logical version. 

3. Nonlogical II. Each of the three learning sequences used 
in this treatment was derived from one of those used in Non- 
logical I by extracting from a Nonlogical I sequence the frames 
belonging to each of the three sections of the original program. 
The frames from each section were grouped together (in their 
Nonlogical I order) and the sections were presented in their 
original 1-2-3 sequence, thus preserving only macro-order. 

4. Nonlogical III. The learning sequence used in this treat- 

ment was prepared by presenting the three sections of the origi- 
nal program in the order 3-1-2. Therefore, multiplication of 
matrices came before any of the preliminary material, thus dis- 
rupting only macro-order. 
A group of 119 eighth-grade students were used as subjects in 
Substudy I. In Substudy II, a group of 111 twelfth graders were 
used as experimental subjects. Although the twelfth graders had 
studied determinants, none of the experimental subjects had 
studied matrix arithmetic in school, and all met the criterion of 
entering behaviors as specified by the program author. 
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A 96-frame linear programed instructional unit on matrix arith- 
metic was selected for use in this study (Ablon, 1962). The 
style of the program was brisk and clear, without the apparent 
redundancy associated with many linear programs. The pro- 
gram was developed in a workshop at the University of Roch- 
ester according to methods suggested by Lysaught and Williams 
(1963), and is considered to be among the best of the many 
programs written at these workshops over the past several 
years. It was field tested successfully by its author, and has 
been used in a study of predicted and actual success of in- 
structional programmers (Lysaught & Pierleoni, 1966). 

A 46-item objective criterion measure, requiring constructed, 
matching, yes-no, and multiple choice responses, was developed 
for use in this study. The reliability of the measure was esti- 
mated to be 0.90 by the Kuder-Richardson Formula Number 21. 
Because other investigators of sequencing had indicated that 
nonlogical sequences appear to challenge some learners more 
than logical sequences (e.g. Roe, Case, & Roe, 1962, p. 104), 
an effort was made to assess change in mood as a result of the 
experimental treatments. For this purpose, a shortened version 
of the Mood Adjective Check List (MACL) (Nowlis & Green, 
1965, p. 111) was used. The list consisted of three adjectives 
on each of nine factors: aggression, anxiety, surgency, concen- 
tration, fatigue, social affection, sadness, egotism, and skepti- 
cism. By asking the subjects to relate each adjective to their 
feelings before and during the experimental treatment, changes 
in mood could be detected. 

Students in each of the substudies were randomly assigned to 
one of the eight treatment combinations. Because it was nec- 
essary to work within the framework of the regular mathe- 
matics class period, Substudy I (eighth graders) was carried out 
on successive days Monday through Friday of one week, and 
Substudy II (twelfth graders) was carried out in a different 
school on successive days from Monday through Thursday of an- 
other week. At no time were experimental materials used out- 
side the classroom. 

Subjects were not pretested on matrix arithmetic because 
pretesting would-have damaged the prior information variable, 
and it was determined that the experimental subjects had not 
studied matrices as a part of their regular curriculum. Immedi- 
ately after instructions were completed on the first day, the first 
MACL was administered to the subjects, who then commenced 
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work on the program. The second MACL was administered at 
the end of the second day in order to assure that all subjects 
would have spent approximately the same amount of time un- 
der the experimental treatment. As soon as each subject finished 
working on the program, he took the criterion measure. Ap- 
proximately one month later, the same criterion measure was 
administered as a measure of retention. At this point, the pur- 
pose of the study was explained to the subjects. 

Only four of the 119 subjects in Substudy I were lost because 
of absence, and only two of 111 in Substudy II. 

Two scores were obtained from the criterion and retention 
measures: the definition-addition score, derived from all items 
covering the definition and addition of matrices (maximum of 
31 points), and the multiplication score, derived from those 
items covering the multiplication of two matrices (maximum of 
26 points). In addition, data were also collected on time spent 
on the program and number of errors on the program. 

These data were analyzed with 2 x 4 analyses of variance, 
with two planned comparisons along the sequencing dimen- 
sion (Hays, 1963, Ch. 14). Comparison 1 was designed to com- 
pare the mean value for the Logical treatment with an average 
of the mean values for all Nonlogical treatments. Comparison 
2 was designed to compare the effect of disrupting micro-order 
with that of disrupting macro-order by comparing means for 
Nonlogical II and Nonlogical III. In order to preserve the or- 
thogonality of the planned comparisons, it was necessary to 
randomly eliminate subjects from certain cells before the anal- 
yses of variance were performed. Any additional comparisons 
suggested by the remainder of the sequencing sum of squares 
were to be performed with a post hoc method, such as the 
Scheffé test (Hays, 1963, Ch. 14). 

Because MACL is a measure of mood change, difference scores 
were obtained by subtracting scores on the first administration 
of the measure from corresponding scores on the second. It was 
felt that a nonparametric (or distribution-free) test would be 
more appropriate inasmuch as these data were considered to be 
ordinal. Therefore, a distribution-free test of analysis of vari- 
ance hypotheses suggested by Wilson (1956) was used to ana- 
lyze the data on each of the nine factors of the MACL. 

The level of significance for rejection of all hypotheses was 
set at .05. 
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Prior information (row) means, sequencing (column) means, 
and the average of all Nonlogical treatment means (used in 
Comparison 1) for Substudy I are presented in Table 1. 


TABLE 1 


Means for Prior Information (Row), Sequencing (Column), and All 
Nonlogical (Comparison 1) Treatments on All Dependent Variables for Substudy I 


Prior Information Means Sequencing Means 
Dependent Prior No Prior Nonlogical All Non- 
Variables Information Information Logical I II III logical 


Criterion Measure 
Definition-Addi- 
tion Score 
(max. 31) 
Multiplication 
Score (max. 26) 
Retention Measure 
Definition-Addi- 
tion Score 
(max. 31) 
Multiplication 
Score (max. 26) 
Time on Program 
(minutes) 
Errors on Program 


26.75 26.45 26.30 26.74 27.15 26.26 26.71 


13.05 12.75 14.33 12.44 14.07, 10.78 12.43 


25.67 25.04 25.23 25.69 26.27 24.27 25.41 


11.26 12.14 12.19 11.46 12.54 10.54 11.51 


90.51 85.63 88.36 91.38 90.70, 82.31, 88.13 


19.32 19.08 15.004 23.96 22.33¢ 15.70. 20.664 


a. Comparison 2; F = 4.52, df = 1,100 
b. Comparison 2; F = 5.99, df = 1,96 
c. Comparison 2; F = 4.18, df = 1,99 
d. Comparison 1; F = 6.59, df = 1,99 


Sequencing 


With respect to the sequencing variable, no significant differ- 
ences were found in the definition-addition portion of either the 
criterion or retention measures. The general level of perfor- 
mance on both was satisfactory, although it can be seen that 
the mean scores for the retention measure were three to five per- 
cent lower than those on the criterion measure. 

The mean scores for all Nonlogical treatments were lower 
than those for the Logical treatment on the multiplication por- 
tion of both the criterion and retention measures, but these dif- 
ferences were not.significant. 

On the multiplication portion of both measures, it should be 
noted that subjects in the two treatments in which macro-order 
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was preserved (Logical and Nonlogical II) performed at a high- 
er level than those in treatments in which macro-order was dis- 
rupted. On the criterion measure, subjects using the program 
in which macro-order had been preserved, but micro-order dis- 
rupted (Nonlogical II) performed at a significantly higher level 
than those using the program in which macro-order had been 
disrupted, but micro-order preserved (Nonlogical III) (see Table 
1). This difference was also found on the retention measure, but 
was not significant. 

Subjects in Nonlogical III made significantly fewer errors 
and took significantly less time on the program than those in 
Nonlogical II (see Table 1). Also, subjects in the Logical treat- 
ment made significantly fewer errors than those in all Nonlogi- 
cal treatments (see Table 1). In general, it can be seen from 
Table 1 that subjects using programs in which micro-order 
was disrupted spent more time and made more errors on the 
program than those using programs in which micro-order was 
preserved. No significant differences ascribable to sequencing 
were found on any of the factors of the MACL. 

With respect to the prior information variable, the only sig- 
nificant difference found was on the sadness factor of the MACL. 
Subjects receiving the prior information became more sad than 
those not receiving this information (x? = 6.93, df = 1). On the 
criterion and retention measures, subjects who received the 
prior information performed at a level only slightly higher 
than those not receiving the information. 

Finally, with respect to the interaction of the sequencing and 
prior information variables, the only significant difference 
found was on the anxiety factor of the MACL (x? = 8.48, 
df = 3). 

Mean scores for this substudy are presented in Table 2. 

With respect to the sequencing variable, subjects using the pro- 
gram which disrupted both macro- and micro-order (Nonlogi- 
cal I) performed at a significantly higher level on the defini- 
tion-addition portion of the criterion measure than those using 
the program in which only macro-order was disrupted (Non- 
logical III) (see Table 2). On the same portion of the retention 
measure, those using the program in which macro-order was 
preserved, but micro-order disrupted (Nonlogical II) performed 
at a significantly higher level than those using a program in 
which only macro-order was disrupted (Nonlogical III) (see 
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TABLE 2 
Means for Prior Information (Row), Sequencing (Column), and All 
Nonlogical (Comparison 1) Treatments on All Dependent Variables for Substudy II 
Prior Information Means Sequencing Means 
Dependent Prior No Prior Nonlogical All Non- 
Variables Information Information Logical I II III logical 
Criterion Measure 
Definition-Addi- 
tion Score 
(max. 31) 30.13 30.00 30.11 30.63, 29.93 29.59, 30.05 
Multiplication 
Score (max. 26) 19.93 20.30 21.93 19.93 21.07p 17.52, 19.50 
Retention Measure 
Definition-Addi- 
tion Score 
(max. 31) 28.95 28.86 29.00 29.08 29.84. 27.72. 28.88 
Multiplication 
Score (max. 26) 16.53 17.31 18.28 16.68 17.49 15.28 16.48 
Time on Program 75.06 79.27 
(minutes) 71.60, 81.46 81.00 74.31 78.924 
Errors on Program 10.22 10.93 6.53erg 13.74, 12.19, 9.744, 11.89 


mo O.0 of p 


. post hoc comparison; Scheffé test with upper limit 1.899 and lower limit 0.181 
. Comparison 2; F = 4.92, df = 1,100 

. Comparison 2; F = 6.58, df = 1,92 

. Comparison 1; F = 5.38, df = 1,95 

. Comparison 1; F = 26.26, df = 1,100 

. post hoc comparison; Scheffé test with upper limit 5.605 and lower limit 0.615 


g. post hoc comparison of Logical and Nonlogical III with Nonlogical I and Nonlogical II; Scheffé 
test with upper limit —3.014 and lower limit —6.546 


Table 2). Nonlogical III registered the largest decrease from 
criterion to retention measure, but all scores were at a high 
level. 

On the multiplication portion of both the criterion and reten- 
tion measures, subjects in the Logical treatment performed at a 
higher level than those in all Nonlogical treatments; these dif- 
ferences, however, were not significant. On the criterion mea- 
sure, subjects in Nonlogical II performed significantly better 
than those in Nonlogical III (see Table 2). This difference per- 
sisted in the retention measure, but was not significant. For 
the multiplication portion of both measures, subjects using pro- 
grams in which macro-order was preserved (Logical and Non- 
logical II) performed at a higher level than those using pro- 
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grams in which macro-order was disrupted (Nonlogical I and 
Nonlogical III). Scores on this portion of the retention mea- 
sure were about 15 percent lower than those on the criterion 
measure. 

Subjects in the Logical treatment made significantly fewer 
errors and took significantly less time on the program than 
those in all Nonlogical treatments (see Table 2). Also, subjects 
in Logical made significantly fewer errors than those in Non- 
logical III (see Table 2). Those subjects using programs in 
which micro-order was preserved (Logical and Nonlogical III) 
took less time and made fewer errors than those using programs 
in which micro-order was disrupted (Nonlogical I and Nonlogi- 
cal II); this difference was significant only for the error data 
(see Table 2). No significant differences ascribable to sequenc- 
ing were found on any factors of the MACL. 

With respect to the prior information variable, the only sig- 
nificant difference found was on the skepticism factor of the 
MACL. Subjects who received the prior information showed a 
significant decrease in skepticism compared to those who did 
not receive the information (x? = 4.52, df = 1). On the defini- 
tion-addition portion of both the criterion and retention mea- 
sures, subjects receiving the prior information performed at a 
slightly higher level than those not receiving the information; 
on the multiplication portion of both measures, the former sub- 
jects performed at a slightly lower level than the latter. 

Finally, with respect to the interaction of the prior informa- 
tion and sequencing variables, no significant differences were 
found on any of the dependent variables. 

In both substudies, the treatment groups that registered the 
better performances on the multiplication portion of both the 
criterion and retention measures were those in which macro-or- 
der was preserved (Logical and Nonlogical II). It would appear 
to be beneficial, therefore, to have some knowledge of what a 
matrix is before one attempts to learn how to multiply two of 
them. This would seem to be consistent with Gagné’s (1962) 
notion of the existence of hierarchies of learning tasks. In order 
to achieve a specific overall objective, there may be subordi- 
nate tasks which must be accomplished in a particular order. In 
both substudies, the significant superiority of Nonlogical II 
(which preserved macro-order but disrupted micro-order) over 
Nonlogical III (which disrupted macro-order while preserving 
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micro-order) on the multiplication portion of the criterion mea- 
sure lends additional support to Gagné’s ideas, and further, sup- 
ports the suggestion by Williams (1966) that the attainment of 
individual tasks within a hierarchy can be accomplished in a 
number of ways, including nonlogical programed sequences. 
These results suggest that there exists a point beyond which 
fastidious attention to logical sequencing will not return divi- 
dends in program effectiveness proportionate to the effort 
expended. 

Also, subjects using the nonlogical sequences performed as 
well as those using the original logical sequence with respect 
to the definition and addition of matrices, tending to confirm 
suggestions by other investigators in this area that students can 
rearrange information effectively. A tentative explanation for 
this behavior can be extracted from the work of Bousefield and 
others (see for example, Bousefield, 1953) on a phenomenon 
called clustering as it relates to Hebb’s (1949, pp. 95-98) ac- 
count of superordinate and subordinate perceptions. 

The following paragraphs suggest answers to the questions 
posed earlier in this paper; the numbers of the paragraphs cor- 
respond to those of the questions. 

1. With respect to the definition and addition of matrices, the 
nonlogical sequences worked as effectively as the logical se- 
quences. Students apparently were able to overcome any diffi- 
culties caused by the disruption of micro-order through some 
means of mental reorganization of the information. Such a 
means was suggested, based on the theoretical work of Hebb 
and Bousefield. With respect to the multiplication of matrices, 
disruption of micro-order again failed to interfere with the sat- 
isfactory acquisition of the information. However, disruption 
of macro-order did interfere with satisfactory acquisition. 

The same trends observed in the criterion measure persisted 
in the retention measure. No sequencing treatment appeared to 
affect retention in any unusual way. The general conclusion 
reached in this study is much the same as the suggestion by 
Hamilton (1964)—sequence of frames does not make a differ- 
ence as long as the order of concepts is preserved. In particular, 
if certain tasks are subordinate to some higher level task, these 
tasks must be accomplished in the proper sequence. As long as 
this condition is met, the actual sequence of frames may not be 
of importance. 
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2. Disruption of micro-order was more costly in terms of time 
spent on the program, and number of errors on the program. 
If both achievement and efficiency in terms of time and errors 
are important, the original logical sequence must be judged the 
best in this study. If time and errors are not of importance, 
both the original program and the nonlogical sequence in 
which macro-order was preserved but micro-order disrupted ap- 
peared to produce comparable achievement. 

3. The availability of prior information apparently was of no 
help to the subjects. Mager’s (1962) suggestion that one might 
need to do little more than give students a list of objectives 
was not supported in this study. 

4. Although there would seem to be little doubt that exposure 
to a learning sequence would have some effect on a student's 
mood or feelings, practically no significant changes in mood 
were detected in this study. Aside from a significant decrease 
in skepticism for subjects who received prior information in 
one of the substudies, there were no other effects of interest at- 
tributable to the experimental variables. 


5. The results of both substudies were strikingly similar. The 


overall lower level of performance of the junior high school 
students was attributed to their lower level of mathematical at- 
tainment. Also, it was concluded that the original program 
could have been more effective in teaching multiplication of 
matrices to the junior high school students. 
The results of this study would not seem to justify the construc- 
tion of linear programed sequences in an intentionally nonlogi- 
cal way; logical sequence still appears to be the best in terms of 
overall effectiveness and efficiency. On the other hand, these re- 
sults do question the necessity for laboring’ over their construc- 
tion, using rigorous methods of content sequencing such as 
mathematics. It would seem to be a profitable expenditure of 
time to employ such methods only to the point of assuring that 
all learning tasks subordinate to the principal task at hand are 
accomplished in the proper sequence. As indicated earlier, it 
would seem to be of value to identify the levels at which sequenc- 
ing is important, so that the effort expended in constructing a 
learning program can be directed towards doing the most good. 
Indeed, one could go on to question the value of a strictly 
linear program. Such programs make no provision for correct- 
ing learner errors, or accommodating individual differences. Be- 
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cause it seems reasonable to assume that individual learners 
will bring more or less unique backgrounds and requirements 
to a given learning situation, the work to be done in the near 
future with sophisticated branching techniques and computer- 
assisted instruction should be of considerable interest to those 
concerned with the most effective sequencing of instructional 
content. 
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